Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KNOX-2936 - knoxcli convert-topology doesn't work with output-path parameter #770

Merged
merged 1 commit into from
Jul 17, 2023

Conversation

zeroflag
Copy link
Contributor

@zeroflag zeroflag commented Jul 12, 2023

What changes were proposed in this pull request?

While validating knoxcli.sh convert-topology found that the cli fails with error saying invalid operation mentioned though provided valid options.

 bash-4.4$ /home/knox/knox/bin/knoxcli.sh convert-topology --path "path/to/topology.xml" --provider-name my-provider.json --descriptor-name my-descriptor.json --topology-name topologyName --output-path "path/to/configs/"
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
KnoxCLI {cmd} [options]
   [--help]
   [version]
   [create-master [--force] [--master mastersecret] [--generate]]
   [create-cert [--force] [--hostname h]]
   [export-cert [--type PEM|JKS|JCEKS|PKCS12]]
   [create-alias aliasname [--cluster clustername] [ (--value v) | (--generate) ]]
   [create-aliases --alias alias1 [--value value1] --alias alias2 [--value value2] --alias aliasN [--value valueN] ... [--cluster clustername] [--generate]]
   [delete-alias aliasname [--cluster clustername]]
   [list-alias [--cluster clustername]]
   [redeploy [--cluster clustername]]
   [list-topologies]
   [validate-topology [--cluster clustername] | [--path "path/to/file"]]
   [user-auth-test [--cluster clustername] [--u username] [--p password] [--g]]
   [system-user-auth-test [--cluster clustername] [--d]]
   [service-test [--u username] [--p password] [--cluster clustername] [--hostname name] [--port port]]
   [list-registry-clients]
   [list-provider-configs --registry-client name]
   [upload-provider-config providerConfigFile --registry-client name [--entry-name entryName]]
   [list-descriptors --registry-client name]
   [upload-descriptor descriptorFile --registry-client name [--entry-name entryName]]
   [delete-provider-config providerConfig --registry-client name]
   [delete-descriptor descriptor --registry-client name]
   [get-registry-acl entry --registry-client name]
   [convert-topology --path "path/to/topology.xml" --provider-name my-provider.json [--descriptor-name my-descriptor.json] [--topology-name topologyName] [--output-path "path/to/configs/"] [--force] [--cluster clusterName] [--discovery-url url] [--discovery-user discoveryUser] [--discovery-pwd-alias discoveryPasswordAlias] [--discovery-type discoveryType]]
   [generate-jwk [--jwkAlg HS256|HS384|HS512] [--saveAlias alias] [--topology topology]]


convert-topology --path "path/to/topology.xml" --provider-name my-provider.json [--descriptor-name my-descriptor.json] [--topology-name topologyName] [--output-path "path/to/configs/"] [--force] [--cluster clusterName] [--discovery-url url] [--discovery-user discoveryUser] [--discovery-pwd-alias discoveryPasswordAlias] [--discovery-type discoveryType]:


Convert Knox topology file to provider and descriptor config files
Options are as follows:
--path (required) path to topology xml file
--provider-name (required) name of the provider json config file (including .json extension)
--descriptor-name (optional) name of descriptor json config file (including .json extension)
--topology-name (optional) topology-name can be use instead of --path option, if used, KnoxCLI will attempt to find topology from deployed topologies.
	 if not provided topology name will be used as descriptor name
--output-path (optional) output directory to save provider and descriptor config files
	 if not provided config files will be saved in appropriate Knox config directory
--force (optional) force rewriting of existing files, if not used, command will fail when the configs files with same name already exist.
--cluster (optional) cluster name, required for service discovery
--discovery-url (optional) service discovery URL, required for service discovery
--discovery-user (optional) service discovery user, required for service discovery
--discovery-pwd-alias (optional) password alias for service discovery user, required for service discovery
--discovery-type (optional) service discovery type, required for service discovery

How was this patch tested?

$ convert-topology --path conf/topologies/sandbox.xml --provider-name my-provider.json --descriptor-name my-descriptor.json --topology-name sandbox.xml --output-dir "/tmp/test"
$ cat /tmp/test/my-descriptor.json 
{
  "provider-config-ref" : "my-provider",
  "services" : [ {
    "params" : { },
    "name" : "NAMENODE",
    "urls" : [ "hdfs://localhost:8020" ]
  }, {
    "params" : { },
    "name" : "JOBTRACKER",
    "urls" : [ "rpc://localhost:8050" ]
  }, {
    "params" : { },
    "name" : "WEBHDFS",
    "urls" : [ "http://localhost:50070/webhdfs" ]
  }, {
    "params" : { },
    "name" : "WEBHCAT",
    "urls" : [ "http://localhost:50111/templeton" ]
  }, {
    "params" : {
      "replayBufferSize" : "8"
    },
    "name" : "OOZIE",
    "urls" : [ "http://localhost:11000/oozie" ]
  }, {
    "params" : {
      "replayBufferSize" : "8"
    },
    "name" : "WEBHBASE",
    "urls" : [ "http://localhost:60080" ]
  }, {
    "params" : {
      "replayBufferSize" : "8"
    },
    "name" : "HIVE",
    "urls" : [ "http://localhost:10001/cliservice" ]
  }, {
    "params" : { },
    "name" : "RESOURCEMANAGER",
    "urls" : [ "http://localhost:8088/ws" ]
  }, {
    "params" : { },
    "name" : "DRUID-COORDINATOR-UI",
    "urls" : [ "http://localhost:8081" ]
  }, {
    "params" : { },
    "name" : "DRUID-COORDINATOR",
    "urls" : [ "http://localhost:8081" ]
  }, {
    "params" : { },
    "name" : "DRUID-BROKER",
    "urls" : [ "http://localhost:8082" ]
  }, {
    "params" : { },
    "name" : "DRUID-ROUTER",
    "urls" : [ "http://localhost:8082" ]
  }, {
    "params" : { },
    "name" : "DRUID-OVERLORD",
    "urls" : [ "http://localhost:8090" ]
  }, {
    "params" : { },
    "name" : "DRUID-OVERLORD-UI",
    "urls" : [ "http://localhost:8090" ]
  }, {
    "params" : { },
    "name" : "HUE",
    "urls" : [ "http://localhost:8889" ]
  } ],
  "applications" : [ ]
}
$ cat /tmp/test/my-provider.json 
{
  "providers" : [ {
    "role" : "authentication",
    "name" : "ShiroProvider",
    "enabled" : "true",
    "params" : {
      "sessionTimeout" : "30",
      "main.ldapRealm" : "org.apache.knox.gateway.shirorealm.KnoxLdapRealm",
      "main.ldapContextFactory" : "org.apache.knox.gateway.shirorealm.KnoxLdapContextFactory",
      "main.ldapRealm.contextFactory" : "$ldapContextFactory",
      "main.ldapRealm.userDnTemplate" : "uid={0},ou=people,dc=hadoop,dc=apache,dc=org",
      "main.ldapRealm.contextFactory.url" : "ldap://localhost:33389",
      "main.ldapRealm.contextFactory.authenticationMechanism" : "simple",
      "urls./**" : "authcBasic"
    }
  }, {
    "role" : "identity-assertion",
    "name" : "Default",
    "enabled" : "true",
    "params" : { }
  }, {
    "role" : "hostmap",
    "name" : "static",
    "enabled" : "true",
    "params" : {
      "localhost" : "sandbox,sandbox.hortonworks.com"
    }
  } ]
}

@zeroflag zeroflag self-assigned this Jul 17, 2023
Copy link
Contributor

@moresandeep moresandeep left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! thanks for catching it!

Copy link
Contributor

@smolnar82 smolnar82 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@zeroflag zeroflag merged commit 82a31c5 into apache:master Jul 17, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants